“What You can See Is What You can Feel.” -Development of a Visual/Haptic Interface to Virtual Environment-

In this paper, we propose a new concept of visual/haptic interfaces called WYSIWYF display. The proposed concept provides correct visual/haptic registration using a visionbased object tracking technique and a video keying technique so that what the user can see via a visual interface is consistent with what he/she can feel through a haptic interface. Using Chroma Keying, a live video image of the user’s hand is extracted and blended with the graphic scene of the virtual environment. The user’s hand “encounters” the haptic device exactly when his/her hand touches a virtual object in the blended scene. The first prototype has been built and the proposed concept was demonstrated.